<!DOCTYPE html>
<html>

  <head>
    <meta charset='utf-8'>
    <meta http-equiv="X-UA-Compatible" content="chrome=1">
    <meta name="description" content="btlike BT搜索引擎 : 项目文档">

    <link rel="stylesheet" type="text/css" media="screen" href="stylesheets/stylesheet.css">

    <title>btlike BT搜索引擎</title>
  </head>

  <body>

    <!-- HEADER -->
    <div id="header_wrap" class="outer">
        <header class="inner">
          <a id="forkme_banner" href="https://github.com/btlike">View on GitHub</a>

          <h1 id="project_title">btlike BT搜索引擎</h1>
          <h2 id="project_tagline">项目文档</h2>

        </header>
    </div>

    <!-- MAIN CONTENT -->
    <div id="main_content_wrap" class="outer">
      <section id="main_content" class="inner">
        <h3>
<a id="在线演示" class="anchor" href="#%E5%9C%A8%E7%BA%BF%E6%BC%94%E7%A4%BA" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>在线演示</h3>

<ul>
<li><a href="http://demo.btlike.com">demo.btlike.com</a></li>
</ul>

<h3>
<a id="特性" class="anchor" href="#%E7%89%B9%E6%80%A7" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>特性</h3>

<ul>
<li>高性能: 5美金的单核,768MB的<a href="https://www.vultr.com/pricing/">VPS</a>上，每秒处理UDP请求超过12K，内存占用不超过100MB，每天抓取数千万去重infohash</li>
<li>大容量: 采用Mysql分表存储,分16张表,设计容量6千万~8千万数据</li>
<li>全文搜索：采用<a href="https://github.com/elastic/elasticsearch">elasticsearch</a>实现全文索引和热度排序</li>
</ul>

<h3>
<a id="组件" class="anchor" href="#%E7%BB%84%E4%BB%B6" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>组件</h3>

<ul>
<li>
<a href="https://github.com/btlike/spider">spider</a> 底层DHT网络爬虫库</li>
<li>
<a href="https://github.com/btlike/crawl">crawl</a> 从DHT网络中抓取活跃的infohash并存储</li>
<li>
<a href="https://github.com/btlike/storage">storage</a> 根据infohash从网络中抓取torrent并存储metadata</li>
<li>
<a href="https://github.com/btlike/api">api</a> 对外提供接口，响应搜索等请求</li>
<li>
<a href="https://github.com/btlike/site">site</a> web网站</li>
</ul>

<h3>
<a id="安装准备" class="anchor" href="#%E5%AE%89%E8%A3%85%E5%87%86%E5%A4%87" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>安装准备</h3>

<ul>
<li>安装golang</li>
<li>安装mysql</li>
<li>安装<a href="https://github.com/elastic/elasticsearch">elasticsearch</a>
</li>
</ul>

<h3>
<a id="安装组件" class="anchor" href="#%E5%AE%89%E8%A3%85%E7%BB%84%E4%BB%B6" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>安装组件</h3>

<ul>
<li>crawl<br>
go get github.com/btlike/crawl</li>
<li>api<br>
go get github.com/btlike/api</li>
<li>storage<br>
go get github.com/btlike/storage</li>
<li>site<br>
git clone <a href="https://github.com/btlike/site.git">https://github.com/btlike/site.git</a>
</li>
</ul>

<h3>
<a id="流量图" class="anchor" href="#%E6%B5%81%E9%87%8F%E5%9B%BE" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>流量图</h3>

<p><img src="http://77g42f.com1.z0.glb.clouddn.com/flow.jpg" alt="flow"></p>

<h3>
<a id="设计思路" class="anchor" href="#%E8%AE%BE%E8%AE%A1%E6%80%9D%E8%B7%AF" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>设计思路</h3>

<ul>
<li>从全网尽可能多的获取infohash</li>
<li>到资源库查询而不是DHT网络，采用流量+带宽暴力遍历</li>
<li>极力优化内存、CPU与带宽资源</li>
<li>随着热门资源大部分被收录，每日更新资源数从上百万逐步降低到十来万</li>
</ul>

<h3>
<a id="帮助与联系" class="anchor" href="#%E5%B8%AE%E5%8A%A9%E4%B8%8E%E8%81%94%E7%B3%BB" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>帮助与联系</h3>

<p>在项目下提issue或联系<a href="mailto:yanyuan2046@126.com">yanyuan2046@126.com</a></p>
      </section>
    </div>

    <!-- FOOTER  -->
    <div id="footer_wrap" class="outer">
      <footer class="inner">
        <p>Published with <a href="https://pages.github.com">GitHub Pages</a></p>
      </footer>
    </div>

    

  </body>
</html>
